36 research outputs found
Neural Weight Search for Scalable Task Incremental Learning
Task incremental learning aims to enable a system to maintain its performance
on previously learned tasks while learning new tasks, solving the problem of
catastrophic forgetting. One promising approach is to build an individual
network or sub-network for future tasks. However, this leads to an ever-growing
memory due to saving extra weights for new tasks and how to address this issue
has remained an open problem in task incremental learning. In this paper, we
introduce a novel Neural Weight Search technique that designs a fixed search
space where the optimal combinations of frozen weights can be searched to build
new models for novel tasks in an end-to-end manner, resulting in scalable and
controllable memory growth. Extensive experiments on two benchmarks, i.e.,
Split-CIFAR-100 and CUB-to-Sketches, show our method achieves state-of-the-art
performance with respect to both average inference accuracy and total memory
cost
Activity recognition from videos with parallel hypergraph matching on GPUs
In this paper, we propose a method for activity recognition from videos based
on sparse local features and hypergraph matching. We benefit from special
properties of the temporal domain in the data to derive a sequential and fast
graph matching algorithm for GPUs.
Traditionally, graphs and hypergraphs are frequently used to recognize
complex and often non-rigid patterns in computer vision, either through graph
matching or point-set matching with graphs. Most formulations resort to the
minimization of a difficult discrete energy function mixing geometric or
structural terms with data attached terms involving appearance features.
Traditional methods solve this minimization problem approximately, for instance
with spectral techniques.
In this work, instead of solving the problem approximatively, the exact
solution for the optimal assignment is calculated in parallel on GPUs. The
graphical structure is simplified and regularized, which allows to derive an
efficient recursive minimization algorithm. The algorithm distributes
subproblems over the calculation units of a GPU, which solves them in parallel,
allowing the system to run faster than real-time on medium-end GPUs
Recommended from our members
Live human-robot interactive public demonstrations with automatic emotion and personality prediction.
Communication with humans is a multi-faceted phenomenon where the emotions, personality and non-verbal behaviours, as well as the verbal behaviours, play a significant role, and human-robot interaction (HRI) technologies should respect this complexity to achieve efficient and seamless communication. In this paper, we describe the design and execution of five public demonstrations made with two HRI systems that aimed at automatically sensing and analysing human participants' non-verbal behaviour and predicting their facial action units, facial expressions and personality in real time while they interacted with a small humanoid robot. We describe an overview of the challenges faced together with the lessons learned from those demonstrations in order to better inform the science and engineering fields to design and build better robots with more purposeful interaction capabilities. This article is part of the theme issue 'From social brains to social robots: applying neurocognitive insights to human-robot interaction'.EPSR
Automatic Context-Driven Inference of Engagement in HMI: A Survey
An integral part of seamless human-human communication is engagement, the
process by which two or more participants establish, maintain, and end their
perceived connection. Therefore, to develop successful human-centered
human-machine interaction applications, automatic engagement inference is one
of the tasks required to achieve engaging interactions between humans and
machines, and to make machines attuned to their users, hence enhancing user
satisfaction and technology acceptance. Several factors contribute to
engagement state inference, which include the interaction context and
interactants' behaviours and identity. Indeed, engagement is a multi-faceted
and multi-modal construct that requires high accuracy in the analysis and
interpretation of contextual, verbal and non-verbal cues. Thus, the development
of an automated and intelligent system that accomplishes this task has been
proven to be challenging so far. This paper presents a comprehensive survey on
previous work in engagement inference for human-machine interaction, entailing
interdisciplinary definition, engagement components and factors, publicly
available datasets, ground truth assessment, and most commonly used features
and methods, serving as a guide for the development of future human-machine
interaction interfaces with reliable context-aware engagement inference
capability. An in-depth review across embodied and disembodied interaction
modes, and an emphasis on the interaction context of which engagement
perception modules are integrated sets apart the presented survey from existing
surveys
Audio-driven Robot Upper-body Motion Synthesis
Body language is an important aspect of human communication, which an effective human-robot interaction interface should mimic well. The currently available robotic platforms are limited in their ability to automatically generate behaviours that align with their speech. In this paper, we developed a neural network based system that takes audio from a user as an input and generates upper-body gestures including head, hand and hip movements of the user on a humanoid robot, namely, Softbank Robotics’ Pepper. The developed system was evaluated quantitatively as well as qualitatively using web-surveys when driven by natural speech and synthetic speech. We particularly compared the impact of generic and person-specific neural network models on the quality of synthesised movements. We further investigated the relationships between quantitative and qualitative evaluations and examined how the speaker’s personality traits affect the synthesised movements
Self-Supervised Prediction of the Intention to Interact with a Service Robot
A service robot can provide a smoother interaction experience if it has the
ability to proactively detect whether a nearby user intends to interact, in
order to adapt its behavior e.g. by explicitly showing that it is available to
provide a service. In this work, we propose a learning-based approach to
predict the probability that a human user will interact with a robot before the
interaction actually begins; the approach is self-supervised because after each
encounter with a human, the robot can automatically label it depending on
whether it resulted in an interaction or not. We explore different
classification approaches, using different sets of features considering the
pose and the motion of the user. We validate and deploy the approach in three
scenarios. The first collects natural sequences (both interacting and
non-interacting) representing employees in an office break area: a real-world,
challenging setting, where we consider a coffee machine in place of a service
robot. The other two scenarios represent researchers interacting with service
robots ( and sequences, respectively). Results show that, even in
challenging real-world settings, our approach can learn without external
supervision, and can achieve accurate classification (i.e. AUROC greater than
) of the user's intention to interact with an advance of more than s
before the interaction actually occurs.Comment: Paper under revision for Robotics and Autonomous Systems journa
Learning to Solve Tasks with Exploring Prior Behaviours
Demonstrations are widely used in Deep Reinforcement Learning (DRL) for
facilitating solving tasks with sparse rewards. However, the tasks in
real-world scenarios can often have varied initial conditions from the
demonstration, which would require additional prior behaviours. For example,
consider we are given the demonstration for the task of \emph{picking up an
object from an open drawer}, but the drawer is closed in the training. Without
acquiring the prior behaviours of opening the drawer, the robot is unlikely to
solve the task. To address this, in this paper we propose an Intrinsic Rewards
Driven Example-based Control \textbf{(IRDEC)}. Our method can endow agents with
the ability to explore and acquire the required prior behaviours and then
connect to the task-specific behaviours in the demonstration to solve
sparse-reward tasks without requiring additional demonstration of the prior
behaviours. The performance of our method outperforms other baselines on three
navigation tasks and one robotic manipulation task with sparse rewards. Codes
are available at https://github.com/Ricky-Zhu/IRDEC